Learning Fast Bipedal Locomotion Master’s Thesis

In this thesis, I present a method to optimize the clock-driven, periodical walking pattern of a humanoid robot for forward speed using metaheuristics. I start from a hand-tuned open-loop gait and enhance it with two feedback control mechanisms. First, I employ a P-controller to regulate the foot angle to reduce angular velocity of the robot’s body. The angular velocity is measured by a gyroscope. Second, I propose a phase resetting mechanism that resets the step cycle at the moment of foot contact. In simulated experiments, I demonstrate that feedback control is essential to achieve fast and robust locomotion. To derive an adequate metaheuristic for optimizing the gait, I compare a Downhill Simplex search, a policy gradient reinforcement learning approach (PGRL), and Particle Swarm Optimization. I demonstrate in simulated experiments that PGRL is the most effective algorithm for solving the optimization problem considered in this thesis. I extend the PGRL algorithm by an adaptive step size and a sequential sampling procedure in order to reduce the number of evaluations of the fitness function. Furthermore, I prove that my extensions to the PGRL increase the performance of the algorithm significantly in terms of the quality of the found solutions. I used the extended PGRL algorithm to optimize the gait of a real robot. After optimizing the open-loop trajectory generation parameters and the feedback parameters of the gait, the robot can walk at a speed of 34cm/s. This corresponds to a gain of over 50% compared to the former top speed achieved using a hand-tuned gait.